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ABSTRACT 



This report provides an assessment of the progress of 
Sacramento START, an after-school learning program, for the period from 
September 1996 through May 1997. Evaluation instruments included standardized 
test scores for students in grades three through six and interviews with 18 
principals from 20 START schools and teachers from 18 schools. Test scores 
are reported for the Sacramento (California) City Unified School District, 
the North Sacramento School District, and the Natomas School District. The 
program's goals are to provide a safe, positive afterschool environment for 
low-income children, to build the capacity of these children to succeed 
academically and socially, and to connect neighborhoods with schools by 
providing part-time employment to neighborhood residents, including the 
families of some START participants. The Sacramento START program operated in 
five school districts, and enrolled more than 2,000 students. It employed 134 
staff members, 73% of whom were residents of the neighborhood of the 
participating school. An estimated $7.9 million in low- income family child 
care costs was saved, and over $500,000 was earned by neighborhood residents. 
The cost to the community was estimated at $3.50 per child per day. Minority 
children comprised 83% of participants. In the North Sacramento School 
District, the longer a child was enrolled, the greater the improvement in 
standardized test scores. In the Sacramento City District, 83% of enrolled 
students showed improvement in test scores, and the average improvement was 
an impressive 22 points for third and fourth graders. In the Natomas 
District, 56% of students improved their standardized test scores. Overall, 
62% of teachers thought that START had helped their students academically, 
with improvements most noted for younger students. Principals indicated a 
strong level of support for the program. Recommendations from teachers and 
principals centered on strengthening the relationship between school and 
START staff members, increasing staff training, and focusing more on homework 
and literacy needs of students. An appendix contains nine tables of 
supplemental school information and a list of programs by site. (Contains 23 
tables.) (SLD) 
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Sacramento START 
Evaluation Summary 

This report is intended to provide an assessment of the progress of Sacramento 
START, an after-school learning program, between September 1996 and May 1997. 

It lays a foundation for long term evaluation of the program's impact in achieving its 
goals. Evaluation instruments used in this report included standardized test scores 
for students in third through sixth grades and interviews with principals and teach- 
ers. Test scores are reported for Sacramento City Unified School District, North 
Sacramento School District and Natomas School District only. The evaluation 
process was designed to provide valuable information that can be used to strengthen 
and improve the quality of the program, rather than to make a definitive judgement 
about the impact of the program. A minimum of three years of program operation is 
needed to form the basis for a judgement about the program's performance. 

Program Description, September, 1996-May, 1997 

The Sacramento START program reports the following goals: 1) to provide a safe, 
positive afterschool learning environment for low-income children; 2) to build the 
capacity of these children to succeed academically and socially; and 3) to connect 
neighborhoods with schools by providing part-time employment to neighborhood 
residents, including family members of participating START students. 

The Sacramento START program reported the following data for the 1996/1997 
academic year: 

• Programs operated at twenty elementary schools in five school districts (Sacra- 
mento City Unified, North Sacramento, Natomas, Del Paso and Elk Grove); 

• More than 2,000 students were enrolled two and a half hours a day, four days a 
week; 

• Students received homework assistance, literacy training and enrichment activi- 
ties; 

• Programs were funded through a public/private partnership that included the 
City of Sacramento, five school districts and numerous corporations, foundations and 
individuals; 

• The budget for the school year was $948,000; 

• 134 site staff were employed, 73 percent of whom were residents of the neighbor- 
hood of the START school; 

• Over $500,000 was earned by low-income neighborhood residents through part- 
time employment with Sacramento START; 

• An estimated savings of $7.9 million dollars in school-age child care costs accrued 
to low-income families as a result of the enrollment of their children in START; 

• All children attended free of charge, at a cost to the community of $3.50 per child 
per day; 

• 83 percent of the students enrolled were either Asian-American, Affican-Ameri- 
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can, Hispanic or Latino-American, or Native- American; 

• 58 percent of the START students lived in households in which English was 
not the primary language spoken; 

• 87 percent of the START students qualified for the federally funded free and 
reduced lunch program; 

• 74 percent began the program with reading test scores below the 50th percen- 
tile and 79 percent began with math scores below the 50th percentile. 




START is a family 
activity for Jessica 
Jones, shown here 
with fellow third 
grade START 
students making 
maracas. Her 
parents are both 
program leaders 
and her older 
brother volunteers. 
Jessica made 
substantial gains in 
reading and math 
scores between 
1996 and 1997. 



Summary of Findings: START Students Test Scores Improved 

Participating school districts use different testing systems. To adjust for these 
differences, District's provided scores converted to Normal Curve Equivalent (NCE) 
scores. The NCE is an equal interval, normalized standard score based on national 
test results for the test administered. It is very similar to percentile, but NCE has 
equal intervals between scores while percentiles do not. Scores ranged from 1 to 99, 
have a mean of 50 and a standard deviation of 21. A child that scored 54 NCE on a 
test last year and 54 NCE on a test this year has shown normal academic growth, 
and has maintained the same relative standing to others of his or her grade over 
both years. This is indicated by a zero (0) for NCE change. If the student shows 
greater than normal growth, this will be indicated by a positive number (+1, +2, etc.) 
for NCE change. If the student does not maintain normal growth, this will be indi- 
cated by a negative number (-1, -2, etc.) for NCE change. Testing was done for stu- 
dents in grades three through six. 

Based on test scores adjusted to NCE, the following results were obtained for the 
1996/1997 academic year: 

• In the North Sacramento School District, the longer a student was enrolled in 
START, the greater the improvement; those who had been in START since January 
1996 improved an average of 6 NCE points between Spring 1996 and Spring 1997. 

• Compared with students participating in The Community That Cares programs in 
North Sacramento, START students showed greater improvement in their test 
scores (5.3 NCE compared with 3 NCE). 
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• In the Sacramento City Unified School District, among the lowest reading group 
(below the 26th percentile), 83 percent of the students enrolled in the START pro- 
gram improved dramatically; the average improvement was an impressive 22 points 
for third and fourth grade students and 15 points for fifth and sixth grade students. 

• In the Sacramento City Unified School District, 67 percent of the START stu- 
dents improved their NCE scores, with average improvement being 5.4 NCE. When 
compared with their peers who did not have START, the improvement was greater 
for START students. 

• In the North Sacramento School District, 55 percent of the START students 
improved their NCE scores, with the average improvement being 4.6 NCE. When 
compared with their peers who did not have START, the improvement was greater 
for START students. 

• In the Natomas School District, 56 percent of the START students improved their 
NCE test scores, with the average improvement being 4 NCE. 

START Impact as Reported by Classroom Teachers and Principals 

Interviews with a random sample of classroom teachers at 18 schools provided 
insight into the teachers' assessment of START and START's impact on students. 
There is some inconsistency between teacher assessments and test score assess- 
ments, indicating that more thought needs to be given to these measures. Where 
teachers perceived improvement in students' academic or social skills, they were 
likely to attribute at least some of the improvement to START. Teachers of younger 
students and those lacking English skills were more likely to see a START impact on 
the students' skills. 

• Overall, 62 percent of the teachers thought that START had helped their students 
improve academically. 

• Where teachers perceived improvements in homework completion and quality, self 
esteem, social skills and respect for learning, in most cases, the teachers believed 
that START had helped achieve the improvements. 

• 83 percent of the kindergarten students, 75 percent of the first grade students and 
60 percent of the students learning English were judged by their teachers as being 
helped by START to improve academically. 

Interviews with principals indicated a strong level of support for the START pro- 
gram, and a desire to work with the program to improve its impact on students. 

Recommendations by teachers and principals emphasized strengthening the rela- 
tionship between the school staff and the START staff, increasing training for 
START personnel on classroom management and discipline, and focusing more the 
literacy and homework needs of the children enrolled in the program. 

Recommendations 

• Drop-outs do not receive the benefit of the program. While mobility is a 
demographic characteristic in low income areas, significant numbers of START drop- 
outs remained in residence at the school. Further investigation of the drop-out issue 
may help the program improve performance. 
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Introduction 



In 1995, concern about public education in the City of Sacramento launched the City 
into a new direction. Student performance on achievement tests documented that 
most students were performing below the national average. The START program 
was designed by the City to infuse new support into the school environment. Funds 
were allocated and partnership agreements were forged between the City and five 
school districts, Sacramento City Unified School District, North Sacramento School 
District, Del Paso School District, Natomas School District and Elk Grove School 
District. Private sector and community support expanded the START partnership. 

During the period under study, the START goal was to build the capacity of children 
to succeed academically and socially by providing after-school learning experiences 
for nine hours a week to one hundred or more students at each site. In September, 
1996, a pilot evaluation of the first five months of START was completed. Based on 
what was learned during the pilot phase, the evaluation design for this study was 
prepared. This second evaluation study focuses on the first full academic year of 
operation (September, 1996-May, 1997). 

The priority in preparing this report was to focus on test score results. The 
difficulties of working with each district to assess test score results and make 
comparisons absorbed the bulk of the energy devoted to this project. The test score 
data are limited and also cannot measure many of the changes START is seeking. 

Developing the START program to respond to the specific strengths and problems of 
each site is a challenge. Site level personnel need feedback that will help them fine- 
tune their work. It would be helpful to identify site specific activities that are 
exemplary in achieving results. The evaluation process could provide the 
background information that would assist in that process. However, this report does 
not provide site level information. In part this is due to the fact that at the site level 
test score data is available only for a few students at each level. The numbers often 
do not permit generalizations to be drawn. 

This study also did not gather data about the perceptions of parents, students or 
START staff, all of whom have an important role and stake in the program. Ideally, 
an evaluation process would engage all the participants in the evaluation process. 

This study did provide continuous feedback to the START Director as results became 
available. She was able tomake program adjustments and to design the 1997-1998 
program to respond to the evaluation findings. 

This report records the findings of evaluation, and demonstrates that the START 
program had a measurable impact on student performance duirng the 1996-1997 
academic year, and is fulfilling a much needed role in the educational community. 
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issue may help the program improve performance. 

• Competent and dependable volunteers can have a significant impact on program 
performance. The program infrastructure has been provided through public-private 
funding and partnership, but improving the success of the program depends upon 
getting more adults spending more time with more children on a sustained basis. In 
small groups and through one-on-one attention many of the children in the program 
can accelerate their academic growth. 

START should establish volunteer participation goals and monitor progress, with a 
regular report to the community leadership. More media attention to those 
volunteers who exemplify the best qualifications could help. To the maximum extent 
possible, START families must be involved as volunteers. A task force of specially 
qualified community leaders and exemplary START parents specifically focused on 
the goal of reaching volunteer participation goals may be the next step. 

• Collaboration between the START program and the schools is a key element that 
requires on-going and two-way communication. 

• Personnel issues are always very central to the labor-intensive work of teaching. 
START recruits, trains, manages, supervises and evaluates over 100 employees who 
work directly with children. Working with the schools and other partners to 
continuously improve staff skills and effectiveness will obviously be a big part of 
START’s ongoing challenge. 




• Evaluation focusing solely on test score achievements fails to measure important 
elements of the program's achievement. Expanding evaluation tools could provide a 

better picture of START's role in 
the community and with the 
children, and more useful 
feedback about how to improve 
the program. 



Volunteer Virginia Moose works primarily with these 
third grade hoys on their homework: Frederick 
Castle (standing left), Poly Regina and Yong Yi Her 

(right). 





The START Challenge 

The START program seeks to improve academic performance and other student 
skills through after school learning experiences. START is directed at those schools 
with the greatest need. In general, START schools are drawn from the bottom half of 
each District’s schools in terms of academic test performance. Table 1 describes the 
schools with START programs in terms of their reading and math test score 
performances in 1995/96; the percentage of students whose home language is not 
English; the percentage of students who are from Aid to Dependent Children 
families; and in terms of the ethnic mix in the school. The schools do vary somewhat, 
but most are high in poor reading and math scores and most exceed the district 
average for AFDC families. These schools also tend to have more families where 
English is not the language spoken at home. In general, the schools have a mix of 
different ethnic groups, and only in three schools is there a predominant ethnic 
group. 



Table I: 

START Schools, Demographic Characteristics 





Reading 


Math 


Home 








Ethnicity 




Site 


Below 


Below 


Language 












Native 


# 


25th% 


25th% 


Not Enalish 


AFDC 


Asian 


Afr-Amer HiSDanic 


White 


American 


2 


17% 


22% 


15% 


42% 


16% 


36% 


20% 


27% 


1% 


4 


34% 


23% 


66% 


81% 


63% 


15% 


14% 


7% 


1% 


6 


39% 


46% 


28% 


67% 


17% 


47% 


21% 


13% 


1% 


7 


34% 


36% 


25% 


52% 


7% 


15% 


39% 


37% 


2% 


9 


42% 


47% 


25% 


77% 


27% 


21% 


22% 


29% 


2% 


10 


41% 


43% 


66% 


94% 


67% 


14% 


14% 


5% 


1% 


11 


25% 


33% 


23% 


70% 


16% 


27% 


29% 


27% 


1% 


12 


31% 


29% 


50% 


58% 


2% 


17% 


63% 


14% 


4% 


13 


31% 


29% 


45% 


55% 


39% 


16% 


28% 


16% 


2% 


14 


41% 


51% 


54% 


72% 


36% 


19% 


34% 


11% 


0% 


16 


>50% 


>50% 
















17 


>50% 


>50% 
















18 


>50% 


>50% 
















20 


-- 


-- 


6% 


21% 


10% 


30% 


25% 


34% 


1% 


21 


42% 


47% 


49% 


79% 


23% 


25% 


39% 


12% 


1% 


22 


23% 


27% 


48% 


90% 


45% 


29% 


18% 


7% 


1% 


23 


42% 


38% 


53% 


69% 


40% 


16% 


32% 


10% 


3% 


24 


52% 


43% 


56% 


77% 


50% 


28% 


15% 


6% 


1% 



Data on schools obtained from school districts. For SCUSD, consult website at www.scusd.edu/ 
schools. 
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Some START schools have high turnover rates, while others are more stable. High 
transiency contributes to high turnover in the START program itself. Table 2 
describes the START program in terms of enrollments, drop-outs, and completions 
during the period January 1, 1996 to December 31, 1996. (Additional drop-outs were 
recorded between January and May 1997; these are treated as participants in the 
report.) Consult the appendix for a school by school breakdown. 



Table 2a: 

Length of Participation in START 



Category 

Enrolled Jan-Dec.,1996 
Enrolled Jan-May, 1996 
Enrolled Sept.-Dee., 1996 
Dropped during Fall 
Dropped during Spring 
Entered late Fall 
Total 

Table 2b: 

Reasons for Drops 



% of START Students 
14 % 

26% 

29% 

11 % 

13 % 

m 

100 % 



Average # Per Site 
28 

52 

53 
26 
27 
14 

200 



Category 


% of DroDDed Students 


# of DroDDed Students 


Attendance 


60% 


330 


Moved 


14% 


78 


Parent Withdrew Child 


8% 


43 


Transportation 


4% 


22 


Student Withdrew 


4% 


23 


Student Withdrew for Other Program 


4% 


22 


Behavior 


2% 


13 


None 


3% 


16 


Total 


99% 


550 



On average 68 percent of the students enrolled in the START program completed 
one or more semesters by December, 1996. On average, 14 percent completed the 
entire year of the program. Sites varied with one site having less than 50 percent of 
their START students completing one or more semesters. In contrast, three sites had 
more than 80 percent of their START students complete at least one semester. 



The reason given by START site coordinators for most drop-outs was lack of 
attendance. Only 2 percent were expelled for behavior reasons. 
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START’s Impact on Academic 
Performance 

The most objective way to answer whether students are learning more is to compare 
test scores. Test and re-test scores are available for some of the students in the 
START program. There are a number of constraints: 1) students move in and out of 
school; 2) students drop START for other reasons; 3) students graduate, 4) in some 
districts, students are not given standardized tests until they reach third grade; 

5) students may not be given standardized tests if they are not proficient in English 
and 6) students may miss school the day of the test. 




As an example of the ability to measure program impact, consider School A. In 1996, 
182 students were enrolled in the START program at one time or another, and 111 
completed at least three months of the START 
program. 



•24 students (13%) completed 8 months of the 
program 

•39 students (21%) completed the first five months of 
the program only 

•58 students (32%) completed the three months in 
the fall of 1996 only 

For 111 students, there are 30 students, or less than 
one-third, for whom there are two test scores to 
compare. 



A child who scores 54 
NCE on a test last year 
and an NCE score of 54 
again this year has shown 
expected normal 
academic growth, and 
has maintained the same 
relative standing to 
others of his grade over 
both years. 



Although the three districts use different tests, the comparison for all students is the 
Normal Curve Equivalent (NCE) change. The Normal Curve Equivalent is an equal 
interval, normalized standard score and is based on national tests results for the test 
administered. It is very similar to percentile, but NCE has equal intervals between 

scores while percentiles do not. 




Valerie Knight is a sixth grader at American Lakes, 
shown here sharing a story with first grade student 
Cornelius. Valerie made substantial gains in both 
reading and math scores between 1 996 and 7 997 
tests. Cornelius' gains were not measured by the 
standardized testing process. 
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Scores range from 1 to 99, have a mean of 50, and a standard deviation of 21. A child 
who scores 54 NCE on a test last year and an NCE score of 54 again this year has 
shown normal academic growth, and has maintained the same relative standing to 
others of his grade over both years. If the student does not maintain normal growth, 
the NCE score will fall, and if the student shows 
greater than normal growth, his NCE score will be 
greater than last year’s NCE score. A growth of one 
standard deviation or more (21 plus NCE) is a very 
substantial growth. 



Fifty-six percent of the 
students achieved 




Natomas School District 

Natomas School District uses the CTBS4 
testing system, Comprehensive Tests of 
Basic Skills, 4th Edition, and tests are 
given to first through sixth graders. To 
compare student performance improvements from 
Spring, 1996 to Spring, 1997, scores in reading 
comprehension and math concepts and applications 
were compared. Students who were in kindergarten 

or sixth grade in 1996 could not be included in the matched scores analysis. Forty-six 
START students had two tests scores in reading and math. Fifty-six percent of the 
students improved their NCE score between 1996 and 1997, achieving greater than 
expected growth. On average, START students improved 3 Normal Curve 
Equivalents (NCE) in reading and 5 NCE in math. 



greater than expected 
growth. On average, 
START students 
improved 3 Normal 
Curve Equivalents 
(NCE) in reading and 5 
NCE in math. 



Improvement was greatest for 1997-third graders and for 1997-fifth graders. On 
average second and fourth graders improved little or not at all. Table 3 provides the 
average test score improvements by grade. 



Table 3: 

Natomas School District START Students Changes in Test Scores, 
1996 to 1997 (Normal Curve Equivalents) 



Grade 


Readina 


n 


Math 


n 


2nd 


1 


11 


0 


11 


3rd 


12 


15 


12 


15 


4th 


-6 


13 


-1 


13 


5th 


9 


7 


11 


7 


Average 


3 




5 





Limited numbers preclude more detailed quantitative analysis. The site coordinator 
for this START program reviewed these data and commented that there were some 
program characteristics that may have affected the positive results for the third 
graders. First, the third grade students who did improve markedly included students 
with serious academic problems, so the improvements are not because the students 
in this group were exceptionally good. Secondly, she noted that the relationship 
between the START program and the third grade teachers was closer than that 
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relationship with other grade teachers. For one thing, the third grade classrooms 
were used by the START program all year for homework time. During that time, 
teachers were also usually present in the classrooms. Therefore the third grade 
students may have been reinforced by their teachers in their START work. Also the 
teachers were able to maintain better than average communication with the START 
project leaders. The site coordinator also commented that the fifth graders had a 
consistent project leader during the homework time, while the second grade students 
experienced a lot of turnover with their project leaders. Also neither the site 
coordinator nor the project leaders had contact with the second grade teachers. 



North Sacramento School District 

North Sacramento School District uses the CAT5 test system, the 
California Achievement Test. The District compiles a “matched scores 
report” to compare students with test scores from two points in time on 
reading comprehension and math concepts and applications. The score used is the 
normal curve equivalent (NCE) for 1996 and 1997. 




The findings cover 105 third through sixth grade students in the district who had 
exposure to the START program, either in the pilot phase (January - May, 1996), or 
the Fall, 1996 phase, or in both. Only students with 
test scores in both 1996 and 1997 are included. 



Among the START students in North Sacramento 
School District for whom we have matched test scores, 
55 percent improved in their reading test scores (NCEs) 
and 53 percent improved in math. The average reading 
improvement was 4.6 NCE and the average math 
improvement was 4.4 NCE. 

The START student improvement can be compared 
with the district-wide Title V sample student 
improvement of 3 NCE in reading. (See Minicucci 
Associates, The Community that Cares: Second Year 
Evaluation Report, August, 1997.) 



Fifty-five percent 
improved in their 
reading test scores 
(NCEs) and 53 
percent improved in 
math. The average 
reading improvement 
was 4.6 NCE and the 
average math 
improvement was 
4.4 NCE. 



Importantly, the longer the student had been in 

START, the greater the average test score improvement. Those who had been 
in since January, 1996 improved an average of 6 points each on reading and math; 
those who had completed the START pilot, but not re-enrolled improved an average 
of 4 points and those who had enrolled in fall, 1996 and remained at least until 12/ 
20/96, improved an average of 3 points in reading and 2 points in math. 



The District requested that T-Tests be performed on the matched scores. The 
differences in scores for all START students were statistically significant at .01 for 
reading scores and .03 for math scores. The differences for students who were 
enrolled for the full year were also statistically significant (at .03 and .05). For the 
students not enrolled for the whole year, the scores differences would not be 
considered statistically significant. 
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Table 4: 

North Sacramento Average Test Score Improvements (NCE) by Exposure 
to START Program, Statistical Significance Tests 



Enrollment 


Reading 

Chanae 


Reading 

Sianificance 


n 


Math 

Chanae 


Math 

Sianificance 


n 


All START 


5 


0.01 


104 


4 


0.03 


98 


Full program 


6 


0.03 


43 


6 


0.05 


38 


Fall only 


3 


0.21 


31 


2 


0.36 


29 


Pilot only 


4 


0.11 


29 


4 


0.14 


31 



There were differences in performance by grade, with sixth graders showing the 
largest gains. 



Table 5: 

North Sacramento Test Score Improvements for START Students by Grade 



Grade 


Readina 


n 


Math 


n 


3rd 


2 


23 


5 


20 


4th 


0 


30 


4 


31 


5th 


4 


25 


0 


24 


6th 


12 


27 


9 


25 


Average 


5 




4 





Comparing START NSSD Student Score Improvement with Non-START 
Students by Grade 

START students were compared with non-START students at the same schools by 
grade to estimate whether START students had an additional increment of 
improvement which could be attributed to the START program. Overall, START 
improved .75 NCE more than non-START students. START students with a full 
year of the START program improved 1 NCE more than the non-START students. 
START sixth graders (n=27) did remarkably better than their peers who 
did not have START. 






Sacramento City Unified School District 



Sacramento City School Unified District (SCUSD) uses the SALT 
(Sacramento Achievement Levels Test) system, and tested students in 
reading and math in the Fall of 1996 and in the Spring of 1997. The 
District made available data to compare students with test scores from two points 
in time on reading comprehension and math concepts and applications. These data 
were then translated into NCE scores. 



The findings cover 653 third through sixth grade students in the district who had 
exposure to the START program, either in the pilot phase (January - May, 1996), or 
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the Fall, 1996 phase, or in both. Only students with test 
scores in both Fall of 1996 and Spring of 1997 are 
included. In addition, scores were compared for students 
who dropped out of the START program, but who 
remained in the District and recorded scores on both 
tests. Also, comparisons were computed separately for 
only those students who had been enrolled in the START 
program both in the fall and in May, 1997. Finally, 
comparisons were computed for students on Spring 1996 
to Spring 1997 tests for those students who had been 
enrolled in START in the Spring of 1996. The comparisons from these latter two 
analysis are not reported because they were very similar and support the same 
conclusions. Math score analysis is not reported here because of the extensive 
analysis required of reading scores. 



Throughout all of the 
testing, the most 
significant correlation 
found was that the 
lower the student’s 
reading level in the 
initial test, the higher 
the growth, . . . 



Throughout all of the testing, the most significant correlation found was that the 
lower the student’s reading level in the initial test, the higher the growth was likely 
to be between that test and the second test. Students were grouped into three 
categories: 1) those at or below the 25th percentile in reading in the Fall, 1996 SALT 
test; 2) those between 25th and 50th percentiles; and 3) those over the 50th 
percentile. There were significant differences in growth between these three groups, 
and these differences were found in START students and in non-START students. 
What this means is that analysis of test score differences between groups is 
meaningful only when the groups are broken down by reading level and then 
comparisons are made between students at roughly the same level. 




Among all the START students in SCUSD for whom we have matched test scores, 67 
percent improved in their reading test scores (NCEs). The average reading 
improvement was 5.4 NCE. Among the lowest reading group, those who had 
scored less than the 26th percentile in the Fall of 1996, 83 percent improved 
their reading scores. The average 



NCE change for those improving 
in that group was an impressive 
22 points for students in third and 
fourth grades and 15 points of 
students in fifth and sixth grades. 
Averaged over all the START students 
in this lowest reading group, the NCE 
improvement was 16 points. Students 
in higher levels of reading did not on 
average improve their NCE scores 
between September and May. It is the 
large gains by the lowest group that 
makes the average START student 
show an increased NCE score. 



[Additional analysis on the North 
Sacramento District data 
demonstrated a similar though less 
marked pattern.] ways. 



START students in the garden club study 
lavender with Program Leader Lorena Dunham, 
applying reading and math skills in nractinat 
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Table 6: 

Increases in Reading NCE for START Students at SCUSD 



Readina 


All Start 


3rd 


7 NCE 


4th 


10 NCE 


5th 


3 NCE 


6th 


1 NCE 


Average 


5 NCE 



Lowest Fall Reading Group 
Those who improved 96-97 
22 NCE 
22 NCE 
15 NCE 
15 NCE 



Comparing START SCUSD Student Score Improvement with Non-START 
Students by Grade 

Efforts to compare START score improvements with a control group are fraught with 
difficulties. START is not an experiment where students otherwise equivalent in all 
ways are randomly assigned to be in START or not. This section compares the 
START and non-START students, as if they were in all other ways equal, but with 
caution advised. Some of the difficulties identified by the Sacramento City Unified 
School District Accountability Office in making such comparisons are the following: 

1. There is no appropriate control group to compare with; comparison with other 
students at the same grade level, even students at the same reading level and grade 
level, is a crude comparison that can yield inaccurate conclusions. 

2. There are too many variables that affect test performance to be able to isolate and 
precisely measure the impact of START on test performance. Parent support, for 
example, may be a critical intervening variable. 

3. The assignment of students to START and non-START groups is not random, nor 
is it possible within the financial constraints of the program to set up a matched 
pairs comparison that isolates the START impact. 

4. A program should be fully implemented before it is evaluated; it needs two to 
three years to operationalize the concept. It is premature to use the test score 
analysis comparisons for the START program as a way to judge its effectiveness. 

There are other, corollary reasons for not relying upon test score analysis to evaluate 
the START program. First, test score data exist for only a portion of the students, as 
pointed out earlier. Second, the students’ choices about how long to stay in the 
START program further reduce the available pool of students to compare, or, 
alternatively, add the complication of classifying students with very different 
exposures to START in the experimental group. 

Given these concerns, comparison of test score changes in NCE units between 
students at START schools who were involved in the START program for at least 
one semester with those who had no involvement or were early drop-outs, yielded 
interesting results, however. First, the pattern of change in both groups is that 



positive changes are larger for the group at or 
below the 25th percentile in reading in the 
Fall of 1996. In this group, the average change 
for START students was 3.5 NCE greater 
than for non-START students. The difference 
was most pronounced for fifth grade, not 
present for fourth grade, and was 2 NCE for 
3rd and 6th grade. 

Students between the 26th and 50th 
percentiles in reading in the Fall of 1996 
showed little difference in average change in 
NCE (1 NCE), with START students not doing 
as well as non-START students. There were no differences in achievement in the 
highest reading group between START and non-START students. 

Overall, averaging all the grades and levels of reading, START students gained 
5.4 NCE and non-START students gained 3.9 NCE. This indicates that on 
average, students were improving more than normal growth, remembering that 
those with the fewest skills were making the progress, while those in the middle and 
upper ends of the spectrum were not making progress beyond normal growth. 

This should not be considered a hard and fast finding, but rather a working finding, 
indicating that there may be reason to believe that the START program is helping 
students at the lowest level to bring themselves further along, when the schools and 
school programs are also moving them in the same direction. In this case, the fifth 
graders reading below the 25th percentile seemed to show the most movement 
beyond that which would have occurred without START. 





Chloe Martinez, sixth grade, American Lakes School, handing out the Sacramento Bee to fellow START 
journalism club members, and Eian Connor, fourth grade, Bannon Creek School [right] made substantial 
gams in reading and math between tests in I 996 and I 997. The club activities provide practical ways to 
practice and enhance literacy skills. 
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Teachers Evaluate 
the START Program 




The teachers’ evaluation is based on interviews with 33 teachers and their 
comments on 150 students in the START program. Teachers were randomly selected 
from all teachers with at least three START students. Three teachers were sampled 
from each school, and repeated attempts were made to interview any two of these 
three teachers. In a few cases substitutions were made because teachers were on- 
leave or off-track. Two teachers were interviewed at fifteen of the 20 START schools, 
one teacher gave an interview at three schools, and for two schools, no interviews 
were achieved. The teachers interviewed included all grades, K through 6. 

Interviews were conducted primarily in April, 1997, and ranged from 15 minutes to 
90 minutes in length. Teachers were asked general questions about START and they 
were also asked specific questions about their students in START and the program’s 
impact on the students. 

The attempt to incorporate teacher evaluation into the START program through 
person to person interviews, using open ended questions, revealed a broad range of 
responses from the elementary teaching community. Some common themes were 
identified that have been helpful to the START program. But these are interpreted 
against a background of wide variety, and the focus should be on involving teachers 
at the site in on-going collaboration, rather than generalizing from this small 
sample. For this reason, detailed tables are reported in the Appendix. 

Not all teachers are equally available or interested. There was a large contrast in 
teacher response to the request for interview. Some teachers came forward quickly, 
and scheduled time easily and in a place and time where confidential discussion was 
possible. They gave abundantly of their time and shared their views frankly. Their 
good will was remarkably apparent. At the other extreme were teachers who ignored 
many attempts to contact them, some who refused interviews, and others who gave 
very little input when interviewed. In a few cases, teachers scheduled interviews at 
times and places where there was little privacy or opportunity to focus. 

To fully meet its objectives, START should identify teachers in terms of their support 
for the START program, and provide information to them in proportion to their 
interest. START should identify those teachers who are a resource for the program 
and work closely with them while keeping all teachers informed at a basic level 
about the program. A teacher support group could be a very valuable ally in the 
development of the START program, and at least some teachers appear to be very 
willing to provide more support to START if solicited. At the same time, there are 
teachers who will not support the program. Specifically inviting teachers to 
volunteer time to the program could help identify the most supportive teachers who 
could also be delegates for the program in the teacher community at their school. 
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START can gain the confidence of the teaching community by addressing disicpline 
issues, providing sustained and meaningful two-way communication with teachers 
about the program and about their specific students, and by improving focus in 
START on the specific literacy and homework needs of the students. 

Teachers’ Evaluations of START Program 

Most teachers (62%) said that the START program helped 
their students improve academic performance, but 22 
percent could not say whether START had a positive impact 
on academic performance. Only 13 percent said that the 
program did not help their students. One of the constraints 
that teachers faced in assessing the program was their lack 
of involvement and knowledge about what START was 
actually doing with their students. 

Teacher involvement and knowledge about the program was 
fairly low. Just sixteen percent of the randomly selected 
sample of teachers were directly involved in START 
program. Most teachers had some idea of what the START program was about, with 
59 percent describing START accurately. Nevertheless, more than a third, 38 
percent, said they did not know what START does, and 3 percent didn’t answer the 
question (indicating a lack of knowledge). 

Teachers mentioned that they learned from their students about the START 
program, and that they noticed the program when it used their classroom or a 
nearby classroom. A few mentioned that they had gone to observe the START 
program, and others expressed regret that they had not taken the trouble to do so. 
Fifty-nine percent of the teachers reported some verbal contact with the START 
Project Leader for their students, or with the Site Coordinator, while 38 percent 
reported no verbal contact. Thirty-eight percent of the teachers reported some 
written communication from either the Project Leader or the Site Coordinator. 

Teachers were asked to rate the level of communication and teamwork between 
START and what the teacher is doing in the classroom. Twenty-eight percent rated 
it as good, 44 percent said it “needs work” and 19 percent said that teamwork and 
communication “did not function.” Another 9 percent said they didn’t know, or did 
not answer the question. 

These findings indicate that more can be done to increase communication and 
teamwork with teachers. Most teachers are now favorably disposed toward the 
START program, and consider it a positive influence on their students’ academic 
growth. Greater communication and teamwork with teachers can solidify and 
improve the effectiveness of the START program. 

Teachers were asked “What would make it easier for teachers to work with after 
school learning programs?” The most frequent response was to do joint planning 
activities (31%). Teachers were also likely to mention getting feedback from START 
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about how their students are doing (16%). Other kinds of communication were 
mentioned, such as project leader visits to teachers (9%), providing teachers 
information about START (6%), more teacher involvement in START, written 
feedback to teachers, preparation time for START staff, more staff training, START 
student improvement of classroom behavior, Assembly participation, and increased 
focus on goals by START (3% each). Only two teachers said there was no need to 
improve communication and team work, and three teachers said they did not know 
how it could be improved. 

For more detailed information about teachers' assessments of the START program, 
refer to the Appendix. The most prevalent concern of classroom teachers was the 
level of discipline and supervision practiced with START students. An interesting 
contrast was provided by a teacher who had been a START participant in the pilot. 
This teacher explicitly referred to the issue by saying that START should not have 
the same standard of conduct as required during regular school hours. “The kids 
express more of themselves in START and they need that,” the teacher noted. 

Potential conflicts between teacher expectations and START program goals may 
underlie these findings. Some teachers also saw the program as one that should "get 
homework done" or one that would provide one-on-one tutoring for students. The 
program was not intended to fulfill these objectives. 

Teachers’ Evaluations of Students 

Interviews with teachers included a section focused on specific students who had 
been enrolled in the START program. On average each teacher had five students in 
the START program. The sample of 150 students reported here includes all grade 
levels and 18 of the 20 START schools. The teacher was asked whether the student 
had improved during the year in academic performance, homework, self-esteem, 
social skills, other skills, respect for learning and attendance or punctuality. The 
teacher was also asked whether the student had been helped by START to make 
these improvements. 

Interpretation of the results is 
somewhat difficult. Several factors 
are involved: the teacher's knowledge 
of the student's skills, the teacher's 
knowledge of the START program, 
and the teacher's awareness of the 
student's response to the START 
program. It’s important to remember 
that (unlike the pilot evaluation) most 
teachers were not evaluating START 
students “before and after” START. 

Some of the START students were 
enrolled in START during the 
previous academic year, and most had 
been going coterminous with the 
academic year. 




| Mr. Jones and students discuss constitutionals^^ | 
[ rights in journalism club. " ~z j 
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Some students were characterized as “always been good,” or “a good student” . Some 
of these may have been performing well because they were in the START program. 
For other students, the teachers described an array of intervention strategies 
targeted to the student. For some students, teachers identified as the key change 
agent the parents or family members who were working with the students, or time 
spent with resource specialists, tutors, or others. 

In other cases, the teacher reported that the student, despite various strategies for 
assisting him or her, had made no progress. For younger children this may be due to 
immaturity; for older children, it may be due to the fact that they are so far behind 
that they have withdrawn mentally and emotionally from the task. Some children 
have learning disorders; some have problems at home that affect their performance 
at times or consistently. 

While teachers in the Sacramento City Unified School District had test scores for 
students in the fall, other districts tested only in the Spring. Interviews with 
teachers preceded availability of spring test scores, so teachers did not have access to 
this information when providing assessments about student improvement. 
Comparing teacher assessments with test score assessments indicates that there is 
some level of disjuncture. Thirty-one percent of the students who had clearly 
improved test scores in both reading and math over the previous test were not 
considered by teachers to have improved ("did not improve" or "don't know if 
improved"), and 31 percent of the students that teachers did believe had improved 
showed lack of improvement in either or both reading and math scores. 

Teachers were also asked to judge changes in social skills, self-esteem and respect 
for learning. Many of the teachers were able to identify cases where they had 
noticed changes that they attributed to START. Often, however, the teachers did not 
consider their students to be lack ing in these areas and therefore would not have 
noticed any changes that may have been engendered by the START program. 

Table 11 calculates the percentage of children helped by START from a base of 
students that teachers believed had improved during the year. (Those not reported 
improving include both good students and poor students.) Table 11 makes clear that 
where improvement was reported, START helping was also likely to be reported. 



Table 1 1 : 

Teacher Reports on Students and START Impact on Students 



Area of Improvement 


Improved 


START heloed 


Percentaae 


Academic performance 


104 


64 


62% 


Homework completion 


62 


58 


94% 


Homework quality 


70 


58 


83% 


Self-esteem/confidence 


69 


55 


80% 


Social skills 


63 


45 


71% 


Other skills 


46 


37 


80% 


Respect for learning 


32 


26 


81% 



Sixty-two percent of those students judged to have improved academically 
were also believed to be helped by START. Ninety-four percent of the students 
that teachers believed had improved in their homework completion were judged by 
teachers to have been helped by START. Eighty-three percent of the students who 
improved their homework quality were evaluated as helped by START to do so. 

Few students were reported to have punctuality or absentee issues (27 or 2 percent), 
and only 7 percent of these were seen as being helped by START to improve their 
punctuality or attendance problems. If START has helped improve student 
attendance or punctuality, it is not detectable through teacher interviews. Again, if 
students began the school year with the teacher and with the START program, the 
teacher is not in a position to judge whether START has helped the student improve. 

Academic Performance 

As reported earlier, in general 62 percent of the teachers said that they believed 
START had helped their students to improve academic performance. Teachers’ 
assessment of student change and START’s role varied a great deal. 

As Table 12 shows, teachers identified only 22 percent of the students as not being 
helped by START to improve when they needed help. This includes two groups, those 
who did not improve (12%) and those who did improve but the teacher believed it 
was not due to START (10%). For another 23 percent the teacher did not know if 
START had helped (18%), or did not know if the student had improved (5%). For 13 
percent of the cases, students did not improve but they were good students with 
consistently good records. 



Table 12: 

Did START Help the Student Improve Academically? 



Response 


Frequencv 


Percent 


Yes 


62 


41% 


No improvement - good student 


20 


13% 


No improvement - student needs to improve 


18 


12% 


Improved - START did not help 


15 


10% 


Don’t know if START helped 


27 


18% 


Don’t know if student improved 


8 


5% 


Total 


150 


100% 



Not including the “don’t know” responses (35), 62 of the 115 students, or 54 
percent, were considered to have been helped by START to improve, and 17 

percent were considered to have been consistently good students not requiring 
improvement. Twenty-nine percent of this group did not improve or their 
improvement was not aided by START, according to the teacher. 



There were also significant differences between grades and between English and 
English-learning students. 

Teacher assessment of STARTs impact of student academic improvement varied 
significantly by grade. Kindergarten (83%) and first grade students (75%) were most 
likely to be judged as being helped by START academically, and sixth graders (15%) 
were least likely. (Caution is needed in interpreting these data, since some of the test 
score data indicate that older students gained more from START involvement.) Sixty 
percent of the students learning English were judged by their 
teachers as being helped by START in improving academically, 
compared with 38 percent of those who are native English 
speakers or already speak English well. 

While teachers’ assessments are subjective and limited in 
scope, they do provide some clues to how the START program 
is working. From the teachers’ perspective, it tends to be more 
important for younger grades and for those needing additional 
assistance in developing English language skills. These are 
precisely the students for whom test score data is typically 
lacking. It is especially interesting that test score 
comparisons showed significant gains for fifth and sixth 

graders beyond their peers not in START, but fifth and sixth grade teachers were 
less likely to notice and attribute academic improvements to START. 

Homework Completion 

Ninety-four percent of the students who teachers believed had improved in 
homework completion were judged by teachers to have been helped by START (see 
Table 13). In assessing START’s impact on homework completion, teachers noted 
that some students had always completed their homework (36%) and for some 
homework completion had 
not improved during the 
year (19%). Thirty eight 
percent of the students were 
judged to have been helped 
by START to improve 
homework completion. For 
six percent of the students, 
teachers did not know if 
there had been a START 
impact on homework 
completion. 




Kindergarten 
(83%) and first 
grade students 
(75%) were most 
likely to be 
judged as being 
helped by START 
academically. 



f-ur/i grade students Baa Nguyen. Tenisha Clan and Jenny 
Tien, work on homework completion in the START program at 
Jcdcduih Smith School. 
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Table 13: 

Did START Help Improve Homework Completion? 



Response 


Freauencv 


Percent 


Yes 


58 


38% 


No, homework always completed 


54 


36% 


No, homework completion not improved 


29 


19% 


Don’t know if START helped improve 


9 


6% 


Total 


150 


100% 



Teachers who had some involvement in START were more likely to say that students 
had improved homework completion, and more likely to say that START had helped 
students improve homework completion. Involvement typically included use of the 
teacher’s room, often enabling teachers to observe the program. It also included 
three cases of teachers who were part of the START program. In two of the three 
cases, the teachers’ roles were primarily after-school homework sessions. 

According to teacher evaluations, START helped some grades more than others with 
homework completion. The impacts of START on homework completion were 
greatest in the first and second grades. Seventy-five percent of first graders and 56 
percent of second graders were judged helped by START in this way, while only 8 
percent of sixth graders were. Other grades were in-between these extremes. Some 
fifth and sixth grade teachers mentioned that their homework requirements for the 
students exceeded the time that START allotted for homework. One said she did not 
think they should have enrichment activities unless they had completed their 
homework. The START program, however, is not intended to complete student 
homework, but to ensure that children know how to complete their homework. 

Some teachers 
mentioned other 
strategies they had 
for getting students to 
do homework, such as 
disincentives they 
offer (having to miss 
out on fun activities, 
use recess for 
homework) and 
enlisting parent help. 

In some cases, they 
believed it was their 
strategy rather than 
the START program 
that improved their 
completion. 



Sixth grade Jedediah Smith START students Anthony Hoang, Orelia 
Castle and Juan Johnson work on getting homework done. 
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START had a greater impact on English language 
learners in homework completion improvement, with 
45 percent helped as compared to 36 percent of non- 
ELL students. 

There were statistically significant differences 
among sites, indicating that site coordinators may 
have put more emphasis on it at some sites. At one 
school, the site coordinator set up homework return 
boxes so that students were able to return the 
homework to their teacher before leaving school on 
the day it was assigned. A number of teachers 
mentioned that completing homework and turning 
in homework were two different tasks, and that 
sometimes the students left homework at home, or 
otherwise lost it. The ability to turn in homework 
after completing it in START addresses both tasks. 

Homework Quality 




Volunteer Virginia Moose listens 
as Rosemary Coronado reviews a 
homework problem. 



Eighty-one percent of the students that teachers 
believed had improved in their homework quality 

were judged by teachers to have been helped by START (see Table 14). This 
represented 39 percent of all START students assessed by their teachers. For 25 
percent of the students, teachers did not think that START had helped the student 
improve and for 15 percent, the teachers did not know whether START had helped. 
Twenty-one percent of the students had consistently good records on homework 
quality according to the teacher. 



Table 14: 

Did Start Help Improve Homework Quality? 



Response 


Freauencv 


Percent 


Yes 


58 


39% 


Did not 


38 


25% 


Homework quality always good 


32 


21% 


Don’t know 


22 


15% 


Total 


150 


100% 



Teachers who were involved in the START program were more likely to say that 
students had improved in homework quality. They were not statistically more likely 
to say that START helped the student improve homework quality. The impacts of 
START on homework quality were greatest in the first and second grades and least 
in the sixth. Fifty-five percent of the English Language Learners were helped by 
START in improvement of their homework quality. Teachers emphasized that for 
these students, there is usually no one at home who can help them. 
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Self Esteem/Confidence 



Teachers said that START helped 37 percent of the START students improve in self- 
esteem or confidence. These students were eighty percent of the students that 
teachers believed had improved in self-esteem or confidence. In most cases, those 
judged to not have improved in self-esteem were considered already high in self- 
esteem and confidence. 



Table 1 5: 

Did START Help Improve Student Self-esteem or Confidence? 



Response 


Frequency 


Percent 


Yes 


55 


36.7% 


Did not 


70 


46.7% 


Don’t know 


24 


16.0% 


Total 


150 


100.0% 



Impacts on self-esteem appear to be greatest for Kindergarten (75%), first grade 
(56%), sixth grade (46%), and second grade (42%); less impact was identified for fifth 
grade (28%), third grade (24%), and fourth grade (21%). 

From teacher's responses, there was no differential impact of START on English 
language learners improvements in self-esteem and confidence. 

Social Skills 

Thirty percent of the START students were believed by teachers to have been helped 
by START to improve social skills. These were seventy-one percent of those students 
judged to have improved in social skills. 



Table 16: 

Did START Help Improve Social Skills? 



Response 


Frequency 


Percent 


Yes 


45 


30% 


Did not 


79 


52% 


Don’t know 


24 


16% 


No response 


2 


m 


Total 


150 


100% 



Impacts on social skills appear to be greatest for Kindergarten (83%), first grade 
(75%), sixth grade (38%), and second grade (31%); less impact was identified for fifth 
grade (17%), third grade (15%) and fourth grade (14%). Kindergarten and first 
graders were much more likely to be helped by START in social skills according to the 
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teachers. There was no differential impact of START on English language learners 
improvements in social skills. 

Other Skills 

For 25 percent of the students, the teacher identified skills that START had helped 
the student improve. ELL students were more likely to improve other skills than 
other students because language was one of the skills noted by teachers as being 
improved by START. START appears to have had impact on other skill development 
for sixth, fourth, Kindergarten, first, second, and fifth grades in that order. 

Respect for Learning 

Teachers identified only 17 percent of the students for whom the START program 
seemed to have improved respect for learning. Teachers identified 38 percent of the 
students as already having high respect for learning. Students most likely to be 
helped by START to increase respect for learning were Kindergarten students (50%) 
and first graders (42%). 

Punctuality and Attendance 

From the teachers' perspective, START had least impact on punctuality or 
attendance. Few students were reported to have these problems (18%), and those 
who did were not considered by their teachers to have been helped by START to 
improve on punctuality or attendance. Often teachers attributed these problems to 
problems at home. 

Drop-Outs 

Teachers were also asked to comment on students who dropped out of START. 
According to teachers' responses, 81 percent of the drop students needed a program 
like START, and only 19 percent were expected by teachers not to need START for 
academic improvement. For 28 percent of the drop-outs, teachers did not know why 
the student had dropped. For the other cases, there were a variety of reasons given. 
The largest single group of reasons had to do with children's reasons for not wanting 
to go to the program. These included the desire to play more, feelings of frustration 
or boredom, "too long of a day", other activities, including being with relatives who 
had dropped the START program, or shyness or fear. In other cases, the child was 
"needed at home" or parents wanted the child at home. In a few cases, the parents 
were not happy with the START program, or were disciplining the child by taking 
away the START program. Transportation and family schedule problems also 
interfered with START attendance. In a few cases, the child was not going to START 
because of behavior problems or attendance problems. It appears from the teachers' 
responses that the START drop-out rate could be improved through stronger ties 
between the family and the START program. 
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Evaluator’s Comment 

Teachers’ evaluations of the START program are limited by 
their knowledge of the program, their awareness of how well 
students were doing before joining START, and by teachers’ expectations. Our pilot 
evaluation report had found that teachers assessments did vary sometimes from test 
score results. A comparison during this evaluation indicated that about 30 percent of 
the teachers' evaluations about academic performance were at odds with test score 
data. 

The teacher interviews have been useful for providing qualitative information about 
students and for learning more about how teachers view the START program. It is 
not clear that they can provide useful judgements on how well students have grown 
in social skills, self-esteem and self-confidence, or in attitudes toward learning. 
Moreover, many teachers showed reluctance to spend time providing interview or 
questionnaire data. An independent evaluation of student growth in these 
dimensions would be preferable. 

Teachers’ views about the impact of the START program on students are generally 
positive. However, teacher awareness and understanding of START’s impact could 
improve through working more closely with START on diagnosis and attention to 
learning needs of each student. 





Two views of the same classroom, with one 
program leaden Julian Navarette, 
background with baseball cap . and one 
volunteer (background, above). Students 
include Joaquin Sanchez (l) and Hector 
Ibarra (r) above and Nai Saechao (l) and 
Sabrina Love ( r ) in the photo on the right. 
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Principals Evaluate 
the START Program 

Personal interviews were conducted in April, 1997 with 18 principals of the twenty 
schools with START programs Principals were asked about their general 
impressions of the START program, and also about specific strengths and 
weaknesses of the program, at their site, and overall. They were asked about the 
level of teamwork and communication between the START program and the school, 
about the problems (if any) with the START program at their 
site and overall, and asked what changes they would like to 
see in the START program. Principals also provided 
information about the impacts (if any) of the START program 
on parent involvement, on school efforts to reduce graffiti, 
vandalism, crime and absenteeism, and on school efforts to 
improve academic performance, student skills and confidence. 

In general, the principals’ evaluation of START was positive. 

Principals also had many suggestions about ways to improve 
the START program and its relationship to the school. 

General 

Of the eighteen START principals interviewed, 72 percent responded with primarily 
positive statements when asked to give their general impressions of the START 
program. Two principals (11%) were preoccupied with the start-up problems they 
had been experiencing in getting the program underway. Three of the principals 
(17%) were primarily concerned with disappointments they had experienced with 
the START program. [Two principals declined to be interviewed.] 



“It is a needed 
service; It’s 
well managed; 
and we have a 
good site 
coordinator.*’ 

—principal 




Table 18: 

Principals’ General Impressions of START 



Impression 
Primarily positive 
Start-up problems 
Disappointing results 
Total 



Frequency Percent 

1 3 72 % 

2 11 % 

3 17 % 

18 100 % 



The major factor in the principals’ assessments seems to be the site coordinator. 
Staff quality (including turnover) is also a big factor. One principal said, “It is a 
needed service; it’s well managed; and we have a good site coordinator.” Another 
observed, “It is far better than last year. The site coordinator is more effective and 
has worked out the bugs.” One of the disappointed principals said, “There have been 
personnel problems, high turnover, and the quality is lacking.” A principal who 
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strongly supports the program noted, “I was more pleased with last year’s staff. They 
were more experienced and had site experience.” 

Another, more elusive quality, is the connectedness between the school’s academic 
performance goals and strategies and the START program. Where these are 
integrated, principals have high enthusiasm. Where they are not, though the 
principal may support the START program, there is a less positive evaluation. One 
principal observed, “I don’t think it is making a change in scores. I’d like to have a 
START program but with a more academic focus.” On the opposite side, another 
principal said “I like the synergy developing with the mission of this school and the 
community.” Similarly, a positive principal said “It does a real good job of supporting 
the academic program, enhancing literacy and homework. ... we have parents who 
are unable to help their students either because of language or dysfunctionality. 
These kids benefit from START.” 



Strengths of the START Program 

There were many strengths mentioned by the principals during the interviews. 

Personnel. In general, principals were pleased with the personnel operating the 
START program, and often mentioned their site coordinator as a strength of the 
program. In particular they pointed out (where relevant) the importance of having 
bilingual aides from the school as staff in the START program. Principals 
appreciated having school staff and members of the community working as staff for 
the START program. Some noted staff stability as a strength, and some noted 
involvement of certificated staff in the program as a strength. 

Typical responses about strengths of the program were: 

“They are good 
coordinators with 
good skills; they are 
trained.” 

“They are well 
organized. The staff 
is responsible and 
caring.” 

“The personnel 
are community 
people. They know 
the school and the 
students.” 

“Training for the 
staff.” 
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Linkage with school program. Principals appreciated the linkage of the START 
program with the regular school program, regardless of the level of partnership that 
had been achieved. One principal said “We are making inroads to merging the 
systems; we are making a step to link after-school and regular day programs.” 

Another said that a major strength of START is “being able to help students with 
literacy, and homework in small groups; it helps self-esteem.” 

Variety of strengths. There were many individual strengths mentioned by 
principals. Some referred to concepts; others referred to nuts and bolts. Other 
examples include: site based management; back-up plans for staff absence; it is a “safe 
haven for needy students;” there is sensitivity to cultural 
diversity; summer session; use of building/campus reduces 
vandalism; parent support; planned activities; support from 
the City Council; media coverage; snacks for the kids; 
volunteers; involvement of high school students; students 
positive about the program; potential for growth. 

Weaknesses of the START Program 

While principals usually strongly supported the program, most 
also could identify weaknesses in the program. Concerns about 
staff were very important. Just as the staff represented one of 
the most important strengths, staff issues represented an 
important area of weakness. This simply recognizes again that 
from the principal’s point of view, the staff are a defining factor 
for the START program. Twelve of the principals mentioned some staff issue, 
including qualifications, training, absenteeism, turnover. 

In commenting on weaknesses, one principal said the major weakness was 
“inexperience of staff. They need training in basics, in procedures, and they need to 
know the school rules.” For this principal, communication between the program and 
the school was a related concern.Another principal put it this way: “staffing; they are 
part-time staff and they lack commitment. There is absenteeism.” 

Seven of the principals mentioned behavior management, supervision of students, 
and related issues like noise, cleanliness and discipline problems. These principals 
often pointed to a connection between the training of staff and the weakness in 
student supervision. 

One principal observed when thinking about START’s weaknesses: “discipline and 
student behavior — the training of the project leaders. The students are not showing 
enough respect for the START staff.” Another said “the program leaders need to be 
better trained for discipline and management. There is not enough supervision.” 

Principals were also concerned about the linkage of the START program to the 
academic program, to academic performance, to homework completion, to the 
classroom teachers, and the literacy program. Six of the principals expressed concern 
about the academic component. 



“Inexperience 
of staff - they 
need training 
in basics, in 
procedures, 
and they need 
to know the 
school 
rules.”-* 

principal 
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At the other end of the spectrum, three principals mentioned space problems as 
weaknesses. The blend of concerns expressed by principals, ranged from operational 
problems like keeping the school staff up to date on START student drops, to big 
picture issues like the strengthening of the academic performance of the school 
through the academic component of START. 

Teamwork and Communication of the START Program 

Principals were asked about the level of teamwork and 
communication between the START program and their school. 

In general, principals reported strong teamwork and 
communication. The largest group (nine, or 50%) said that 
teamwork and communication were excellent or very good, 
while 6 principals (33%) said that it needed work. Only one 
principal said that teamwork and communication did not work 
at all. Two principals did not answer the question. 

In most cases the principals described successful efforts to 
overcome difficulties in communication. They also made 
practical suggestions on how communications and teamwork 
could be improved at their school. 

A principal who said communication and teamwork are excellent noted: “we address 
issues at staff meetings and the site coordinator keeps me informed.” At this site, the 
site coordinator is also a member of the school’s staff. 

A principal who said that the teamwork “needs work” said “I did not know what the 
schedule is for START; yesterday they had a parent meeting, but I did not know. 
Therefore I could not help them with attendance through my channels of 
communication.” At this school, START had just begun and start-up problems were a 
part of the difficulties identified by the principal. 

A principal who did not answer the question made the observation that 
“communication with the principal was strong; the classroom teachers have 
complaints about the use of their rooms, and the custodians have complaints about 
trash and spills.” At this school, conflict between the principal and teachers spanned 
a number of issues, and START may have been swept into a more general lack of 
teamwork and communication. 

Practical suggestions that a number of principals endorsed focused on a need for 
protocols for keeping principals informed. A bulletin or forms submitted on a regular 
basis should inform principals of such things as days when START would not be 
held, names of students dropped from START, events scheduled for parents, 
informing parents when START students are dropped. 



“We address 
issues at staff 
meetings and 
the site 
coordinator 
keeps me 
informed.” — a 
principal 
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START Impact on Graffiti, Vandalism, Absenteeism and Discipline 

Principals were asked to comment on the problems they have with graffiti, 
vandalism, absenteeism and discipline, and to assess whether START has had any 
impact on those problems at their sites. 



Table 19: 

Principals* Assessments of Graffiti, Vandalism, Absenteeism and Discipline 



Problem Area 


No Problems 


Some Problems 


N/A 


Graffiti 


39% 


61% 




Vandalism 


44% 


54% 




Absenteeism 


39% 


39% 


22% 


Discipline 


28% 


54% 


18% 



Principals varied in their assessment of the incidence of graffiti, vandalism, and 
discipline problems and the extent to which absenteeism is a problem at the school. 
None of the principals reported serious or intractable problems. Among those who 
did report problems, most reported that no improvement had been achieved because 
of START. However several principals said that they liked having the program on 
campus after school because they believed it could be helping to protect the campus. 
Without the START program on campus, vandalism and graffiti could be worse. 



Table 20: 

Principals* Assessments of Changes in Graffiti, Vandalism, Absenteeism 
and Discipline 



Problem Area 


Same 


Improved 


Graffiti 


91% 


9% 


Vandalism 


80% 


20% 


Absenteeism 


71% 


29% 


Discipline 


67% 


33% 



Discipline and absenteeism were most likely to change during this period, but 
principals generally did not consider START as a part of what was causing the 
change. Regarding students who are discipline problems, four principals said that 
some of the students with discipline issues were in START while three said that the 
students with discipline problems did not stay in START. 

Principals were asked to list the other programs they had in place to address iissues 
like literacy, vandalism, graffiti, discipline, self-esteem and pride in the school. The 
following list was generated during principal interviews. None of the following 
programs does what START does; and none has the scope that START has. 

However, they represent corollary efforts going on that also may have positive 
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impacts in the same direction as intended by START. Any gross level measurements 
of change would include any independent effects of these programs as well. 



Table 2 1 : 
Programs in Place 



Class Size Reduction 
Student Council 
Caught Ya Being Good 
Monthly Awards 
Literacy Programs 
Exchange Program 
Girl Scouts 

Title IV Safe and Drug Free Schools 
Title I Intersession 
Serve Sacramento 
PTA 

Tutoring Programs 
Homework Club 
SMART (tutoring) 

Group Reading 
Epstein Model 
Parent Reading Program 
Community Police Officers 



Project Achieve 

Conflict Management Training and Mediation Training 
MESA 

Buddy Program 

Magnet School 

Tutorial Reading 

Visions - a counseling program 

Honor Guard 

Odyssey of the Mind 

Red Ribbon Week 

Extended Day (voluntary teacher program) 

Student of the Month 

Safe Schools 

Beautification Day 

Readers & Writers Workshop 

Chapter I Computer Writing to Read 

4th R 



At each site there is a unique blend of programs underway that together with 
START are directed toward not only better academic performance, but better social 
skills, allegiance to the school, and the reduction of absenteeism, truancy, vandalism 
and discipline problems. In this context, it is not possible to independently assess the 
impact of START. 

Problems and Changes for the START Program 

While principals strongly endorse the concept of extended day programs, and have 
positive opinions about the START program, and many principals would like to 
have many, if not all, of their students in after-school learning programs, they did 

identify problems and desired changes. 
Principals identified a broad range of 
concerns that they had about the 
START program. They also were 
generous with suggestions about how 
to improve the program. These 
comments were offered in the spirit of 
collaboration. In dealing with 
problems, principals usually described 
the kinds of problems that they had 
had, and the solutions that they had 
applied. 
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Volunteer Recruitment and 
Retention 



A major part of the concept of the START program is the involvement of the 
community and neighborhood volunteers. The START program is staffed at a ratio of 
20 students per program leader, but would like to provide a ratio of 10 students per 
adult, or less, at least part of the time. It is also desirable that volunteers be 
available to the children on a regular basis. Ideally, a volunteer would participate at 
least a couple of hours a week for at least six weeks. To achieve an ideal volunteer 

participation rate would have required about 1,200 
hours per week of volunteer effort over the school 
year. That is, at each of 20 schools, there would have 
been four days a week in which five adults provided 
three hours of volunteer service. For a 36 week 
school year the total hours would be 43,200. (The 
actual number of volunteers needed to reach such a 
goal would depend on the average hours per week of 
volunteer service.) 



During the pilot period, START reported 203 
volunteers and did not provide records on 
participation by week. During the 1996-97 academic 
year, START reported 227 volunteers, and an 
additional 60 parents at one school available on call 
to participate in special events, and an additional 59 
Learning and Self Esteem Program high school 
students. These LSEP students committed to a 15 
week program to work with three kindergarten or 
first grade students on a daily basis (Monday 
through Thursday). [This student program added 
approximately an additional 3,480 volunteer hours 
over the period September 1996 to May 1997, or 
about 97 hours per week.] Data in Table 23 does not 
include the Learning and Self Esteem Program and 
is taken from the START Volunteers 1997 Report, June 19, 1997. Table 23 describes 
volunteer participation. 

As Table 23 illustrates, 151 START volunteers with scheduled hours per week gave 
an estimated 492 hours of volunteer service per week. Another 77 volunteers 
provided additional support. The most important service, scheduled participation, 
averaged 3.3 hours a week for 151 volunteers. Adding Learning and Self-Esteem 
volunteers at 97 hours per week equals a total of 589 hours of scheduled volunteer 
involvement with START students. While START is getting closer to its volunteer 
participation goal, it is still far from an ideal volunteer program. 
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START volunteers 
do good, feel good 
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Bee Columnist Diana Griego 
Erwin has written several 
columns urging readers to 
volunteer with START. 
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Table 22: 



Principals' Report of Most Noticed Problems (multiple responses) 



Problem Noticed 

Communication with teachers 

Protocols for communication with school/community 

Need to improve academic content 

Problems with teachers in sharing space 

Personnel qualifications 

Training for START personnel 

Coordination with sports & other programs on campus 



# of Principals 



6 

6 

5 

5 

4 

4 

4 



Suggestions for Change 

1) Protocols for Communication. Several principals made suggestions about how 
to ensure that everyone is kept informed about the START program. Both weekly 
forms and bulletins were mentioned. In particular, principals want to know that they 
and the parents are well informed about days when START is not meeting, what 
students have been dropped from START, and any events scheduled for family or 
other visits to the START program. Some principals specifically want to know that 
there is a procedure for bringing in substitutes for START project leaders, and they 
want to know who the substitutes are. 

2) Linkage with the Classroom Teacher. Principals had a variety of suggestions 
about how to improve the linkage between the START program and the classroom 
teacher. Some strongly preferred a literacy program that would directly reinforce 
classroom teaching. They would like to see START staff trained to support the 
approaches used in the school. Some wanted to see a stronger liaison with the 
teachers, either through regular surveying of teachers about how START students 
are doing in the classroom, or greater START staff time spent meeting with 
classroom teachers. Some principals would like for START staff to spend more time 
on coordination, planning and clean-up. 

3) Discipline and Student Management. Principals would like to see START 
personnel receive more training and coaching in student management and 
discipline. They mentioned the need for maintaining standards of conduct in the 
classroom, and the need for training students in expressing respect. 

4) Build Stronger Relationships with the Community and Parents. Schools 
varied in the degree to which the START program engaged parents and the 
community. Principals recognize the key role of the parents and the community in 
the education of students, and would like to see a greater involvement in the START 
program. Some principals talked about a contract with the parents to engage them 
in some way in the school experience. They strongly recommended that a way be 
found to increase parental commitment to the education process, either through 
volunteering within the START program or in some other way. The need for more 
adults to volunteer within the START program was emphasized as a key to 
improving the students academic performance. 



Table 23: 

START Volunteers 1 996-97 





Number 


Percent 


Percent 






Average 


Scheduled/Tvpe 


Scheduled 


Scheduled 


Total 


Hrs/Wk 


Percent 


Hrs/Wk 


Adults 


82 


54% 


36% 


156.62 


32% 


1.9 


Seniors 


4 


3% 


2% 


11.75 


2% 


2.9 


Corporate 


21 


14% 


9% 


30.38 


6% 


1.4 


Interns (HS & College) 


44 


29% 


19% 


293.25 


60% 


67 


Subtotal 


151 


100% 




492 


100% 


3.3 


Not Scheduled/Tvpe 
Adults 


57 


75% 


25% 








Seniors 


4 


5% 


2% 








Corporate 


10 


13% 


4% 








Interns (HS & College) 


5 


6% 


2% 








Subtotal 


76 


100% 










Total 


227 




100% 









START would need about 400 volunteers willing to work every week of the academic 
year to reach the 1200 hour per week goal. That is a 265 percent increase in 
volunteers. Sixty percent of the scheduled hours of volunteer assistance were 
provided by high school and college interns and 40 percent by adult volunteers. 
While interns typically volunteered 6.7 hours per week, adult volunteers averaged 
far less, with seniors giving more time per week (2.9 hours on average), and 
corporate volunteers giving the least time (1.4 hours 
on average). 



While seniors generally gave more of their time, only 
2 percent of the hours donated were senior volunteer 
hours, and only 4 percent of all volunteers were 
classified as seniors. Corporate participation was also 
relatively low, with only 13 percent of all volunteers 
recruited by corporations and only 6 percent of all 
volunteer hours provided by corporate sponsors. 



START would 
need about 
... a 265 
percent 
increase in 
volunteers. 



Most of the adult volunteers and adult volunteer 
hours appear to have been recruited by the START 
program (internally) and most of these are family, 

friends and neighbors of the START programs at the START sites. START reported 
that 154 of the 227 volunteers (68%) were so recruited. The positive side of that is 
that most of the volunteers are from the families and communities of START 
students. The negative side of that is that the community outside of the START 
program is not generating much support for the program. Of the adults 
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recruited into START as volunteers, 18 (eight percent) were recruited through 
presentations, media or other outreach by the program to the community in general. 
These 18 are all scheduled volunteers and represent 12 percent of the regularly 
scheduled volunteers in the program. 

A much greater response will be needed, particularly from the community outside of 
the START neighborhoods, if the START promise is to be realized. Seniors, in 
particular, retired teachers, and those capable of teaching elementary students at all 
levels, could provide students more attention. While START students are making 
measureable progress with existing volunteers, a greater effort by volunteers could 
significantly accelerate and expand the learning, as well as reach students who 
today are not growing academically. 

Competent and dependable volunteers can have a significant impact on program 
performance. The program infrastructure has been provided through public-private 
funding and partnership, but the success of the program really depends upon 
individuals who are willing to help children. Getting more adults to spend more 
time with more children on a sustained basis seems to be the biggest challenge 
facing START. 

START should establish volunteer participation goals and monitor progress, with a 
regular report to the community leadership. More media attention to those 
volunteers who exemplify the best qualifications could help. To the maximum extent 
possible, START families must be involved as volunteers. A task force of specially 
qualified community leaders and exemplary START parents specifically focused on 
the goal of reaching volunteer participation goals may be a next step. 




Conclusion and 
Recommendations 

During its first full academic year of operation, the START program faced many 
challenges and served many communities. This report demonstrates that the 
START program had a measurable, positive impact on student performance during 
the 1996-1997 academic year, and is fulfilling a much needed role in the educational 
community 

Most START students on average are achieving greater than normal academic 
growth. The program is still in the process of being implemented, and a variety of 
operational issues and challenges must be met to achieve optimum performance. It 
is unrealistic to expect early year operation to have substantial impacts on student 
performance. This report demonstrates that the program has had a measurable 
positive impact on students performing at the lowest levels when compared with 
their peers. It suggests that fifth and sixth graders at low reading levels may be 
taking better advantage of START to improve academically than third and fourth 
graders. 

Recommendations 

• Address Drop-Outs. Drop-outs do not receive the benefit of the program. 
Substantial levels of drop-out from the program impede program 
performance. The program, school, families and community must continue to find 
ways to address the reasons for drop-outs and reduce drop-out from START. 

• Recruit and Retain More Volunteers. Competent and dependable volunteers can 
have a significant impact on program performance. Through additional adult 
attention, children in the program can accelerate their academic progress. 

START should establish volunteer participation goals and monitor progress, with a 
regular report to the community leadership. More media attention to those 
volunteers who exemplify the best qualifications could help. To the maximum extent 
possible, START families must be involved as volunteers. A task force of specially 
qualified community leaders and exemplary START parents specifically focused on 
the goal of reaching volunteer participation goals may be the next step. 

• Collaborate with Supportive Teachers. Collaboration between the START program 
and the schools is a key element that requires on-going and two-way communication. 
Teachers and principals had a number of suggestions on ways to improve the 
collaboration. 

To fully meet its objectives, START should identify teachers in terms of their support 
for the START program, and provide information to them in proportion to their 
interest. START should identify those teachers who are a resource for the program 





and work closely with them while keeping all teachers informed at a basic level 
about the program. A teacher support group could be a very valuable ally in the 
development of the START program, and at least some teachers appear to be very 
willing to provide more support to START if solicited. Specifically inviting teachers 
to volunteer time to the program could help identify the most supportive teachers 
who can also be delegates for the program in the teacher community at their school. 

• Continuous Attention to Personnel Issues. Personnel issues are always very 
central to the labor-intensive work of teaching. START recruits, trains, manages, 
supervises and evaluates over 100 employees who work directly with children. 
Working with the schools and other partners to continuously improve staff skills and 
effectiveness will obviously be a big part of START’s ongoing challenge. 

• Continue and Broaden Evaluation Effort. During the months the research for this 
report was conducted, continuous feedback was provided to the START program. As 
a result of this and other internal evaluation efforts, START incorporated 
suggestions and has made many changes and improvements in the program as 
suggested. A continuous evaluation process assists the program as it grows to 
correct its course in response to feedback. One difficulty is in using scarce 
evaluation resources to cover the many elements worthy of evaluation. This study 
found that using teacher assessments was problematic. By focusing primarily on 
test score data to measure student change, there is a danger that the program will 
not get valuable feedback that would be available from other evaluation tools. More 
thought needs to be given to additional evaluation tools. 
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START Programs by Site, Length of Participation 





Site #: 


2 


z 


18 


4 


6 


17 


9 


Percent of START students: 














Group 1 


Pilot Plus 


13% 


13% 


13% 


10% 


23% 


16% 


11% 


Group 4 


Pilot Only 


34% 


17% 


21% 


24% 


18% 


36% 


40% 


Group 2 


Fall Only 


33% 


17% 


32% 


32% 


19% 


29% 


32% 


Group 3 


Dropped Fall 


14% 


18% 


10% 


18% 


5% 


9% 


9% 


Group 5 


Partial Pilot 


7% 


27% 


13% 


13% 


33% 


3% 


6% 


Group 9 


New 


0% 


8% 


11% 


3% 


3% 


8% 


2% 




Total 


100% 


100% 


100% 


100% 


100% 


100% 


100% 


Number of START students: 














Group 1 


Pilot Plus 


21 


28 


24 


21 


49 


30 


24 


Group 4 


Pilot Only 


56 


38 


39 


69 


40 


69 


83 


Group 2 


Fall Only 


55 


37 


58 


39 


11 


55 


67 


Group 3 


Dropped Fall 


23 


40 


18 


52 


38 


17 


18 


Group 5 


Partial Pilot 


11 


59 


23 


29 


70 


5 


12 


Group 9 


New 


0 


17 


20 


z 


z 


15 


5 




Total 


166 


219 


182 


217 


215 


191 


209 


% in groups 1 , 


, 2, & 4 


80% 


47% 


66% 


65% 


59% 


81% 


83% 


# in groups 1 , 


2, &4 


132 


103 


121 


129 


100 


154 


174 




Site #: 


16 


10 


11 


12 


13 


14 


Averaae 


Percent of START students: 














Group 1 


Pilot Plus 


20% 


16% 


8% 


22% 


7% 


9% 


14% 


Group 4 


Pilot Only 


26% 


18% 


32% 


22% 


28% 


17% 


26% 


Group 2 


Fall Only 


34% 


31% 


27% 


27% 


33% 


29% 


29% 


Group 3 


Dropped Fall 


14% 


11% 


7% 


15% 


8% 


12% 


11% 


Group 5 


Partial Pilot 


n/a 


16% 


15% 


11% 


12% 


19% 


13% 


Group 9 


New 


8% 


8% 


11% 


3% 


13% 


14% 


1% 




Total 


100% 


100% 


100% 


100% 


100% 


100% 


100% 


Number of START students: 














Group 1 


Pilot Plus 


46 


37 


18 


33 


12 


16 


28 


Group 4 


Pilot Only 


60 


42 


71 


33 


49 


30 


52 


Group 2 


Fall Only 


79 


74 


59 


41 


57 


52 


53 


Group 3 


Dropped Fall 


32 


26 


15 


23 


13 


22 


26 


Group 5 


Partial Pilot 


N/A 


38 


33 


17 


20 


33 


27 


Group 9 


New 


18 


20 


24 


4 


22 


24 


14 




Total 


235 


237 


220 


151 


173 


177 


199 


% in groups 1 , 


2, &4 


79% 


65% 


67% 


71% 


68% 


55% 


68% 


# in groups 1 , : 


2, &4 


185 


153 


148 


107 


118 


98 


132 



Group Definitions 

Group 1 Enrolled in December 1996 and was enrolled since at least 3/96 in START 

Group 4 Enrolled for entire pilot period, but not 12/96; rarely, completed some pilot & some fall 

Group 2 Enrolled 1 2/96 since September or October 
Group 3 Enrolled in fall and dropped 

Group 5 Dropped pilot, or added during pilot, and not enrolled 12/96 



Teachers' Interviews: Additional Information 



Teachers were very balanced in their assessment of the START program. They were 
asked to name strengths and weaknesses of the program, and tended, overall, to 
name as many strengths as weaknesses. Four teachers mentioned no strengths and 
five mentioned no weaknesses. 



Table 7: 

Number of Strengths and Weaknesses Mentioned by Teachers 



Number 


Number of Teachers 


Mentioned 


Strenaths 


Weaknesses 


0 


4 


5 


1 


12 


11 


2 


7 


8 


3+ 


10 


9 




33 


33 



Teachers identified a number and a variety of strengths of the START program on 
their campus, and multiple responses were recorded for up the first two of those 
identified. These are listed in Table 8. 



Table 8: 

Strengths of the START Program (Multiple Responses) 





# of 


%of 


% of 


Cateaorv 


Responses 


Responses 


Cases 


The activities 


7 


14% 


22% 


Level of student interest 


7 


14% 


22% 


It’s well organized 


6 


12% 


18% 


Personnel 


6 


12% 


18% 


Help with homework 


3 


6% 


9% 


It’s a safe environment 


3 


6% 


9% 


Student focus is good 


2 


4% 


6% 


The staff care 


1 


2% 


3% 


Momentum 


1 


2% 


3% 


They know our school 


1 


2% 


3% 


Staff/teacher communication 


1 


2% 


3% 


High school tutors 


1 


2% 


3% 


Volunteers 


1 


2% 


3% 


The literacy component 


1 


2% 


3% 


Help for students limited in English 


1 


2% 


3% 


The focus on the individual 


1 


2% 


3% 


The element of celebration 


1 


2% 


3% 
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Teachers were asked what changes they would like to see in the START program 
and up to three changes were coded. Discipline was high on the list, as was 
improvement in teacher/START communication. Teachers also were thinking about 
ways that the START program could increase its academic impact. Some teachers 
were eager to see increases in START capabilities. Table 10 summarizes the multiple 
responses to this question. 



Table 10: 

Changes Teachers Would Like to See in START Program 





# of 


%of 


%of 


Chanae Mentioned 


Responses 


Responses 


Cases 


Discipline 


Practice discipline 


9 


12% 


29% 


Training for START PL in discipline 


3 


4% 


10% 


Teacher/START communication 


Teacher/START meetings 


8 


11% 


26% 


Inform teachers re: students 


6 


8% 


19% 


Inform teachers re: START 


5 


7% 


16% 


Involve teachers 


1 


1% 


3% 


Improve Academics 


Get homework done/check HW 


6 


8% 


19% 


Align START with level review reqts. 


4 


5% 


13% 


More specific help 


(math, English, oral 1 to 1, literacy) 


8 


11% 


26% 


More training for PLs - academic 


4 


5% 


13% 


START Resources 


More staff/more staff hours 


3 


4% 


10% 


Better quality, consistency in staff 


3 


4% 


10% 


More resources 


2 


3% 


6% 


More enrichment/motivational work 


5 


7% 


16% 


Serve more/recruit needy kids 


3 


4% 


10% 


Coordinate with other programs 


2 


3% 


6% 


Transportation 


2 


3% 


6% 


START management improvement 


1 


1% 


3% 


Parent involvement 


1 


1% 


3% 



One positive fifth grade teacher said “Have the teachers be more involved in 
targeting and getting parents involved at the beginning of the year. We need a report 
from START on how our students are doing, like a ‘basketball check-off,’ on behavior, 
attendance and homework. If the numbers get low, ask the teachers who should be 
added to START. The kids really value it.” 

A third grade teacher who has been involved in START said “There needs to be more 
communication with the classroom teacher, and even three-way communication 
involving the parents, such as a report form that all three sign. There should be long 



Parent satisfaction 
Don’t know 
Total responses 



1 

6 

64 



2% 

12% 



3% 

18% 



100 % 

The teachers noted a variety of strengths, and no single element of the program 
attracted a majority of the teachers. The most common responses were either 
positive comments about the activities, or the students’ interest in the activities, or 
were positive comments about the personnel and organization of the program. 



Table 9: 

First Weakness Mentioned for the START Program 



Weakness 


# of Responses 


% of Responses 


Discipline lacking 


9 


27% 


Homework not done 


4 


12% 


Supervision poor 


3 


9% 


Staff Quality 


2 


6% 


Literacy program inadequate 


2 


6% 


Communication needs work 


2 


6% 


Parental involvement low 


2 


6% 


No help for ELL-Spanish 


1 


3% 


YRE Coordination 


1 


3% 


Snack quality 


1 


3% 


Program should serve more 


students 1 


3% 


Don't Know 


3 


9% 


None 


2 


6% 



What is notable in teacher description of weaknesses is that 36 percent of the 
respondents (12 teachers at 9 locations) mentioned either discipline or supervision 
as the weakness of the program. This is was the largest single weakness identified, 
and nothing else comes close. 

Under this heading teachers were concerned about a variety of things. Some had had 
problems with START students using classrooms, including things missing, things 
used, out of order, or misplaced. Some objected to the tone of interaction between 
project leaders and students (lack of respect, lack of decorum appropriate to 
learning, project leaders’ tolerance of behavior that teachers do not tolerate). Others 
complained that school rules were not being enforced or that students weren’t 
managed properly. At its worst, the discipline problem is perceived by teachers as an 
active undermining of their relationship with the student by START project leaders. 

An interesting contrast was provided by a teacher who had been a START 
participant in the pilot. This teacher explicitly referred to the issue by saying that 
START should not have the same standard of conduct as required during regular 
school hours. “The kids express more of themselves in START and they need that,” 
the teacher noted. 






r' 



term and short term goals for each student, with an entry and exit survey of the 
child regarding what they want to achieve and what they did to achieve it.” 

A fourth grade teacher said “there needs to be much more time and focus on helping 
with homework. The parents have complained that the students are not doing their 
homework at START. The management of students to appropriate school behavior is 
also needed.” 

A first grade teacher said “tailor the help to the students that are missing 
something. Ask the teacher what specific skills to work on with individual students.” 

A kindergarten teacher asked for “better alignment with grade level benchmarks 
including rules of conduct. The project leaders should have the districts grade level 
benchmarks. And, we need an in-service day for mutual input.” 
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